Validation Model, NCQA My HEDIS 2023

Antidepressant Medication Management

Author

Affiliation

John Ryan Kivela, MA

The Alliance ACO

Welcome (Bien Venido)

Hello,

This project compares Report Data with Real Data by comparing the Health Choice Value Based Purchasing Quality Roster (VBPQR)¹ for Antidepressant Medication Management (AMM) with Health Choice adjudicated claims. This project is similar in many ways to the VBP Validation Model for Follow Up after Hospitalization (FUH)².

The guidelines for antidepressant medication are far more nuanced³. Nonetheless we have endeavored to define the eligible population using claims that have been processed and paid by Health Choice, and then measuring our Real Data against the VBP Quality Report Data.

The results were surprising! Our research into eligible adjudicated claims identified a similar number of cases to the VBP Quality Roster; however, only 16% of the VBP QR members were validated against adjudicated claims.

We hope you enjoy reading this data story. We are always looking for new collaborators, so please reach out and share your ideas.

Sincerely,

Ryan

The Alliance NCQA HEDIS My2023 Eligibility Model for AMM

The thing about measuring VBP

So, there we were in the first months of the Alliance ACO business, and EVERY provider was concerned that the Value Based Purchasing Quality Roster was inaccurate. Our providers were concerned that their performance scores were underrated, not giving them the credit they deserved.

The Alliance wants its providers to be their best selves and to be recognized. Lack of confidence in the scoring mechanism crowds out enthusiasm and damages commitment to improvement.

So, in January 2023 we developed a validation model for the NCQA measure for Follow Up after Hospitalization (FUH).

The VBP Validation Model FUH7 (2023), Key Takeaways:

• Created a validated data model that can be used to identify eligible member events for FUH7
• Core logic can be used to develop models for other HEDIS/NCQA measures
• Research affirmed providers were underrated for the 2022 measurement year
• Providers deserved a higher score by about 4 percentage points

So what’s next?

That brings us to the new measurement year, January 2023 to December 2023. In this project the VBP Validation Model is going to be applied to the Antidepressant Medication Management (AMM) HEDIS My2023⁴ performance measure.

This model compares Report Data from the VBP Quality Roster with Real Data from adjudicated claims, or claims that have been processed and paid.

The model will answer the following questions:

• Do the results of the Value Based Purchasing Quality Roster (VBPQR) accurately reflect actual adjudicated claims?
• Are the performance scores of Alliance Providers accurate and reliable?

The business objectives are twofold:

• Build an eligibility model for the AMM eligible population.
• Validate the VBP Quality Report using adjudicated claims for eligible cases.

Why invest in this research?

Inaccurate measurement of VBP performance leads to an invalid assessment of the Alliance Providers’ delivery of services to their patients.

Alliance providers were underrated on Follow Up after Hospitalization (FUH7) by at least 4 percentage points for the 2022 measurement year⁵. With the constantly increasing emphasis on payment for quality of care, even small differences in measurement impact overall performance ratings.

Alliance Provider performance scores were underated (red blocks) by at least 4 percentage points in 2022

Underrating provider performance frustrates leadership and dampens the spirits of the clinical teams as they do this very difficult work.

Accurate measurement offers a better opportunity to identify patients in need, maximize performance incentive, and support provider morale.

Setting up the experiment

This manuscript is written using Brisk-DM. Brisk-DM is a structure for doing data science that is based on CRISP-DM⁶, but tailored for executives and business leads. Brisk-DM uses an approachable communication style that is easily understandable to professionals in the workplace.

The current state of things

At the time of this study, The Alliance has assessed the first 3 months of VBP Quality Rosters for the 2023 measurement year. Our application of the VBP Validation Model for FUH7 in June 2023 revealed a similar pattern to the 2022 measurement year where we found that performance scores for FUH were underrated⁷.

The results for FUH are under further investigation while we develop this model for AMM. The recurrence of this issue invites some concern. Learning from last year’s results, we will carefully monitor FUH through the data, and track performance throughout the year, reporting our findings in real time.

Hablemos de dato (Let’s talk about the data)

This data model is simple in concept, but complex in design. The model compares Report Data from the HC VBP Quality Roster with Real Data from Health Choice claims.

What are the data sources?

VBP Quality Roster	Adjudicated Claims
Health Choice receives this data from third party vendor, Cotivity.	A dataset of adjudicated claims queried directly from the HCA data warehouse.
An excel workbook containing a roster of the members deemed eligible for VBP HEDIS NCQA measures including compliance status.	Records are gathered from the Claims and PBM databases, respectively.
The Alliance receives individual rosters for each Alliance Provider.	Claims are extracted for all eligible service codes for the measurement year.

Is the data we use reliable?

VBP Quality Roster	Adjudicated Claims
The outstanding question with the VBPQR is if the underlying data from the third party vendor, Cotivity, is accurate.	Claims data is of the highest quality as it is compiled and reviewed extensively by Health Choice for its own business purposes.
The report quality itself is very high as it is compiled by Health Choice Business Intelligence staff.	The quality of this data is also reviewed by state and federal regulation entities, like AZ Health Care Cost Containment System (AHCCCS).
The underlying data is what is under investigation.

The AMM Eligibility Model

We now embark upon building a dataset of eligible claims for the AMM measure. This process will be significantly more complicated than our previous model for FUH. The description of an eligible member event is very nuanced for AMM.

An eligible member event is defined as follows:

• The individual is an adult, aged 18 or older, and
• A prescription was filled for an eligible antidepressant medication within the “Intake Period” (Index Prescription Start Date (IPSD)), and
• No other prescriptions were filled for eligible antidepressant medications within 105 days earlier than the IPSD (Test for Negative Medication History (NMH)), and
• The individual had a service for at least one of the eligible Major Depressive Disorders (Test for Major Depressive Disorder (MDDx)) within 60 days (-60 days before, or +60 days after), and
• The individual had continuous enrollment with Health Choice (Test for Continuous Enrollment (CE)) for
  • 105 days before the IPSD, and
  • 231 days after the IPSD

Check out the visualization of the AMM Eligibility Model below. There are many components of this model with overlapping time frames. We will describe these steps in further detail in the next section.

AMM Eligibility Model, NCQA HEDIS My2023

You may also think of the criteria as a tumbling bike lock. All four criteria must line up, in the right order, for the lock to open. Once unlocked, the resulting dataset es El Dato Verdad, the Real Data.

AMM Eligibility Model, Primary criteria: 1. Index Prescription Start Date (IPSD) 2. Negative Medication History (NMH) 3. Major Depressive Diagnosis (MDDx) 4. Continuous Enrollment (CE)

The Index Prescription Start Date

Description

The first criteria that must be met is the Index Prescription Start Date (IPSD). This is the date that an individual filled a prescription for one of the HEDIS My 2023 eligible antidepressant medications.

The IPSD is the anchor point for the rest of the eligibility measures. An individual must have an eligible IPSD in order to be assessed against the other tests for this measure.

The member must have an eligible prescription fill for an eligible antidepressant medication between the eligible dates indicated in the visualization above.

Procedure

In order to assess IPSD we did the following:

The 12 month Intake Period begins on May 1 of the year prior to the Measurement Year

• The Measurement Year is: 01-01-2023 to 12-31-2023

• The Intake Period is: 05-01-2022 to 04-30-2023

We extract claims from the PBM database at the Health Choice data warehouse for the given My2023 eligible medications dispensed within the Intake Period.

• The NDC to GPI crosswalk must be used to determine eligible medications. The code to create the crosswalk is in the Appendix.

• The minimum date of the Test for Negative Medication History must also be included in the PBM Claims query. The test for NMH is outlined below.

• The Negative Medication History range is: 01-16-222 to 01-15-2023

• The prescription fill date in this range is called the Index Prescription Start Date (IPSD)

Code

Extract claims data from HC data warehouse

Code

# Declare Date Range for NMH and IPSD range
Declare @start as date = '01-16-2022'
Declare @end as date = '04-30-2023'

# Select variables
SELECT
  pbm.clientID,
  id.PrimaryId,
  pbm.AsOfDate,
  pbm.dtefilled,
  pbm.GpiNumber,
  pbm.GPIClassification,
  pbm.prodname,
  pbm.genericnme,
  pbm.LabelName,
  pbm.GroupId,
  pbm.preslstnme,
  pbm.pbmrxclaimnbr,
  pbm.claimsts,
  pbm.AmtPaidFinal,
  pbm.productid,
  pbm.decimalqty,
  pbm.dayssupply,
  pbm.Gender,
  pbm.birthdte,
  pbm.mbrage

# Identify data sources
FROM
  PBM.dbo.HCICPharmacyClaimSummary pbm
  LEFT OUTER JOIN GlobalMembers.dbo.ClientIdPlus id ON pbm.clientID = id.AzAhcccsId

# Set conditions  
WHERE
  pbm.dtefilled BETWEEN @start AND @end
    AND
    pbm.mbrage >= 18
    AND
    pbm.GpiNumber IN (concatenated_values_GPI)

Conduct IPSD test

Code

# Import the raw claims
PBMClaims <- read.csv("./data/DataRaw_PBMClaims_2023-05-31.csv") %>%
mutate(ProjectId_IPSD = sample(1000000000:9999999999, nrow(.)))



# Convert numeric variable to date
PBMClaims$dtefilled <- as.Date(as.character(PBMClaims$dtefilled), format = "%Y%m%d")

# Convert character variable to date
PBMClaims$AsOfDate <- as.Date(PBMClaims$AsOfDate, format = "%m/%d/%Y")

# write.csv(PBMClaims, "./data/output/PBMClaims.csv")

# Copy data to a new name for the testing
PBMClaims_IPSDTest <- PBMClaims

# Convert the 'dtefilled' column to Date type if it's not already in the correct format
PBMClaims_IPSDTest$dtefilled <- as.Date(PBMClaims_IPSDTest$dtefilled)

# Define the start and end dates for IPSDTestResult
ip_start_date <- as.Date("2022-05-01")
ip_end_date <- as.Date("2023-04-30")

# Create the IPSDTestResult column based on the date conditions
PBMClaims_IPSDTest$IPSDTestResult <- ifelse(
  PBMClaims_IPSDTest$dtefilled >= ip_start_date 
  & PBMClaims_IPSDTest$dtefilled <= ip_end_date, "Valid IPSD", "No Valid IPSD")

# write.csv(PBMClaims_IPSDTest, "./data/output/PBMClaims_IPSDTest.csv")

In lay terms

The provided code is a combination of SQL and R code for data processing and analysis. Here’s a layman’s summary:

1. The SQL code retrieves data from a database table called “PBM.dbo.HCICPharmacyClaimSummary” and joins it with the “GlobalMembers.dbo.ClientIdPlus” table using the client ID. It selects various columns related to pharmacy claim details within a specified date range, and filters for members aged 18 or older and specific GPI numbers.

2. The R code imports raw claims data from a CSV file, converts the date columns to the correct format, and saves the processed data as a new CSV file.

3. The R code then creates a copy of the claims data and adds a new column called “IPSDTestResult” based on date conditions. If the “dtefilled” date falls within a specific range, the result is set to “TRUE”; otherwise, it is set to “FALSE”. The modified data is saved as another CSV file.

The overall purpose of the code is to process and analyze pharmacy claims data, specifically focusing on members who meet certain age and GPI number criteria, and identifying if their claims fall within a specified date range for an IPSD test. (ChatGPT, personal communication, July, 2023).

Test for Major Depressive Disorder

Description

The next stop on the road is to confirm that the individual had a service for at least one of the eligible Major Depressive Disorders (MDDx) within 60 days (-60 days before, or +60 days after) of the IPSD.

A diagnosis of Major Depressive Disorder is identified by cross referencing members with an eligible IPSD and NMH (using PBM records), against behavioral health claims records for MDD. Individual identifying data is used to join PBM records with claims records, and make the comparison.

Procedure

Behavioral health claims are extracted from the HCA data warehouse for all cases where there was an eligible Major Depressive Disorder Diagnosis.

Conduct the MDDx Test by comparing each of the cases identified by PBMClaims with any of the MDDx claims.

The result is a big data solution consisting of over 1.7 million possible combinations. We now have grown our dataset to include an assessment for eligible behavioral health services.

Code

Import the HEDIS My2023 value set for eligible Major Depressive Diagnoses

Code

# Set some things upd for the SQL Query

# Import the Value Set List
ValueSetListMy2023 <- read.csv("./data/ValueSetListMy2023.csv")

# Create a list of the diagnosis codes
MDDxList <- ValueSetListMy2023$Code

# Concatenate for adding to sql
concatenated_values_MDDx <- paste0("(",    ValueSetListMy2023$Code, ")", collapse = ", ")

Query the Health Choice data warehouse for eligible MDDx claims

Code

-- Declare start and end variables
DECLARE @start DATE = '2022-03-22';
DECLARE @end DATE = '2023-10-30';

-- Check if the temporary table exists and drop it if it does
IF OBJECT_ID('tempdb..#ValueSetListMy2023') IS NOT NULL
    DROP TABLE #ValueSetListMy2023;

-- Create a temporary table
CREATE TABLE #ValueSetListMy2023 (Code VARCHAR(100) COLLATE SQL_Latin1_General_CP1_CI_AS);

-- Insert values into the temporary table
INSERT INTO #ValueSetListMy2023 (Code)
VALUES ('101'), ('100'), ('207'), ('116'), ('126'), ('136'), ('146'), ('156'), ('110'), ('120'), 
       ('130'), ('140'), ('150'), ('160'), ('170'), ('190'), ('200'), ('210'), ('1000'), 
       ('213'), ('214'), ('206'), ('202'), ('111'), ('121'), ('131'), ('141'), ('151'), 
       ('211'), ('171'), ('172'), ('173'), ('174'), ('122'), ('132'), ('142'), ('152'), 
       ('112'), ('117'), ('127'), ('137'), ('147'), ('157'), ('119'), ('129'), ('139'), 
       ('149'), ('159'), ('169'), ('219'), ('209'), ('179'), ('199'), ('113'), ('123'), 
       ('133'), ('143'), ('153'), ('203'), ('114'), ('124'), ('134'), ('144'), ('154'), 
       ('204'), ('212'), ('118'), ('128'), ('138'), ('148'), ('158'), ('1002'), ('1001'), 
       ('167'), ('164'), ('191'), ('192'), ('193'), ('194'), ('201'), ('208'), ('F32.0'), 
       ('F32.1'), ('F32.2'), ('F32.3'), ('F32.4'), ('F32.9'), ('F33.0'), ('F33.1'), ('F33.2'), 
       ('F33.3'), ('F33.41'), ('F33.9'), ('14183003'), ('2618002'), ('726772006'), 
       ('320751009'), ('36923009'), ('370143000'), ('1.08111E+16'), ('1.08112E+16'), 
       ('42925002'), ('69392006'), ('63778009'), ('25922000'), ('87512008'), ('79298009'), 
       ('1.6266E+16'), ('40379007'), ('720455008'), ('720454007'), ('720451004'), ('832007'), 
       ('15639000'), ('1.62668E+16'), ('18818009'), ('719592004'), ('720453001'), 
       ('720452006'), ('66344007'), ('38694004'), ('39809009'), ('319768000'), ('71336009'), 
       ('268621008'), ('191610000'), ('191611001'), ('191613003'), ('1.62646E+16'), 
       ('1.62649E+16'), ('1.62648E+16'), ('450714000'), ('73867007'), ('33736005'), 
       ('60099002'), ('75084000'), ('2.51E+11'), ('430852001'), ('77911002'), ('20250007'), 
       ('76441001'), ('1.6267E+16'), ('2.81E+11'), ('28475009'), ('33078009'), ('15193003'), 
       ('36474008'), ('191604000');

-- Query to retrieve data from the claims.dbo.shcavos table
SELECT DISTINCT
    shcavos.primaryID, 
    id.BCBSMedicaidId AS MemberID,
    shcavos.begDate,
    shcavos.PrimaryDiagnosis,
    shcavos.Dx1,
    shcavos.Dx2,
    shcavos.Dx3,
    shcavos.Dx4,
    shcavos.Dx5,
    shcavos.Dx6,
    shcavos.Dx7,
    shcavos.Dx8,
    shcavos.Dx9,
    shcavos.Dx10,
    shcavos.Dx11,
    shcavos.Dx12,
    CASE WHEN v.Code IS NOT NULL THEN 'True' ELSE 'False' END AS MatchFound
    
-- Set data source
FROM claims.dbo.shcavos AS shcavos
LEFT JOIN GlobalMembers.dbo.ClientIdPlus id ON shcavos.primaryID = id.primaryID

-- Go through all of the Dx and look for eligible Dx codes
LEFT JOIN #ValueSetListMy2023 AS v ON shcavos.PrimaryDiagnosis COLLATE SQL_Latin1_General_CP1_CI_AS = v.Code
                                    OR shcavos.Dx1 COLLATE SQL_Latin1_General_CP1_CI_AS = v.Code
                                    OR shcavos.Dx2 COLLATE SQL_Latin1_General_CP1_CI_AS = v.Code
                                    OR shcavos.Dx3 COLLATE SQL_Latin1_General_CP1_CI_AS = v.Code
                                    OR shcavos.Dx4 COLLATE SQL_Latin1_General_CP1_CI_AS = v.Code
                                    OR shcavos.Dx5 COLLATE SQL_Latin1_General_CP1_CI_AS = v.Code
                                    OR shcavos.Dx6 COLLATE SQL_Latin1_General_CP1_CI_AS = v.Code
                                    OR shcavos.Dx7 COLLATE SQL_Latin1_General_CP1_CI_AS = v.Code
                                    OR shcavos.Dx8 COLLATE SQL_Latin1_General_CP1_CI_AS = v.Code
                                    OR shcavos.Dx9 COLLATE SQL_Latin1_General_CP1_CI_AS = v.Code
                                    OR shcavos.Dx10 COLLATE SQL_Latin1_General_CP1_CI_AS = v.Code
                                    OR shcavos.Dx11 COLLATE SQL_Latin1_General_CP1_CI_AS = v.Code
                                    OR shcavos.Dx12 COLLATE SQL_Latin1_General_CP1_CI_AS = v.Code
                                    
-- Set date
WHERE shcavos.begDate BETWEEN @start AND @end

-- Return value
AND CASE WHEN v.Code IS NOT NULL THEN 'True' ELSE 'False' END = 'True';

# Save the result as MDDxClaims_dateslug.csv

Load data from the claims query results into the data model

Code

# Import the MDDxClaims from excel
MDDxClaims <- read_csv("./data/MDDxClaims_20230602.csv")

# write.csv(MDDxClaims, "./data/output/MDDxClaims.csv")

Conduct the MDDx Test

Code

# Step 1: Load the MDDxClaims data
MDDxClaimsTest <- MDDxClaims %>%
  rename(PrimaryId = primaryID)


# Step 2: Add a new column "ProjectId_MDD" with random 10-digit numbers
MDDxClaimsTest <- MDDxClaimsTest %>%
  mutate(ProjectId_MDD = sample(1000000000:9999999999, nrow(.))) |>
  filter(!is.na(begDate)) |> 
  rename("MDDxDate" = begDate)

# Step 2a: Shed a little extra weight
MDDxClaimsTest <- MDDxClaimsTest %>%
  select(PrimaryId, MemberID, ProjectId_MDD, MDDxDate, PrimaryDiagnosis)


# Step 3: Back to the IPSDTest data
# Filter out cases with "No Valid IPSD"
filtered_table <- PBMClaims_IPSDTest %>%
  filter(IPSDTestResult == "Valid IPSD")

# Rename PrimaryId for consistency
# filtered_table <- filtered_table |> 
#   rename(PrimaryId = PrimaryId.x)

# Step 4: Merge PBMClaims_NMHTest and MDDxClaims on PrimaryId, keeping all rows and columns
PBMClaims_MDDxMerge <- merge(x = filtered_table, 
                             y = MDDxClaimsTest, 
                             by = "PrimaryId", 
                             all.x = TRUE)

# Step 4 Rogue: format as date
PBMClaims_MDDxMerge <- PBMClaims_MDDxMerge %>%
  rename(IPSDate = dtefilled)

# Step 4 Rogue: format as date
PBMClaims_MDDxMerge <- PBMClaims_MDDxMerge %>%
  mutate(IPSDate = as.Date(IPSDate),
         MDDxDate = as.Date(MDDxDate))

# Step 5: Determine the number of days between IPSDate and MDDxTestDate as an absolute value
PBMClaims_MDDxTest <- PBMClaims_MDDxMerge %>% 
  mutate(days_between_MDDx = abs(as.numeric(IPSDate - MDDxDate)))

# Step 6: Determine "Positive MDDx" or "No MDDx" based on days_between
PBMClaims_MDDxTest <- PBMClaims_MDDxTest %>%
  mutate(MDDxTestResult = ifelse(days_between_MDDx < 60, "Positive MDDx", "No MDDx")) |>
  filter(!is.na(MDDxDate))

# write.csv(PBMClaims_MDDxTest, "./data/output/PBMClaims_MDDxTest.csv")

In lay terms

The code says something lie this:

The SQL code retrieves data from the “shcavos” table in the “claims” database. It selects distinct records and specific columns including the primary ID, member ID, beginning date, primary diagnosis, and diagnostic codes. The query also checks for matches with specific diagnosis codes by joining with a temporary table called “#ValueSetListMy2023” that contains the desired codes. The resulting table includes an additional column indicating if a match was found for each record.

The R code performs the following operations:

1. Import the MDDxClaims data from a CSV file.
2. Rename the “primaryID” column to “PrimaryId” in the MDDxClaims data.
3. Add a new column “ProjectId_MDD” to the MDDxClaims data with random 10-digit numbers.
4. Filter out rows with missing “begDate” values from the MDDxClaims data and rename the “begDate” column to “MDDxDate”.
5. Select specific columns from the MDDxClaims data.
6. Filter the PBMClaims_NMHTest data to keep only rows where “NMHTestResult” is “New Start” and rename the “PrimaryId.x” column to “PrimaryId”.
7. Merge the filtered PBMClaims_NMHTest and MDDxClaimsTest data on the “PrimaryId” column, keeping all rows and columns.
8. Convert the “IPSDate” and “MDDxDate” columns to Date format.
9. Calculate the number of days between “IPSDate” and “MDDxDate” as an absolute value.
10. Add a new column “MDDxTestResult” based on the condition: if the number of days is less than 60, the value is “Positive MDDx”; otherwise, it is “No MDDx”.
11. Filter out rows with missing “MDDxDate” values from the merged data.

Overall, the code retrieves MDDx claims data, performs data transformations and comparisons, and filters the data based on specific conditions (ChatGPT, personal communication, July, 2023).

Fantastico!

Wow! I can’t believe that we actually pulled that off!

Now we have our confirmation of IPSD and MDDx.

Two down, two to go. Awesome!

Test for Negative Medication History (NMH)

Description

The Test for Negative Medication History determines if an individual is a new recipient of an antidepressant medication. An individual passes this test if they have not had a prescription for an antidepressant medication within 105 days prior to the IPSD.

Be patient. If you are replicating this project, this part of the program takes a minute to run. If you think about what it is doing, it is comparing every row of claims (~147,000 rows) against the IPSD Intake Period range, and for all of the positive returns, comparing each one to each of the the original 147,000 rows for the NMH test.

Think about trying to guess the first 2 numbers on the bike lock, multiplied by 147,000. Anyway, it’s a lot. Go grab some coffee.

Procedure

In order to conduct the test for Negative Medication History we took the following steps:

PBM Claims dataset is queried from the Health Choice data warehouse. This initial pull includes the entire potential range of PBM Claims for the Intake Period and the Negative Medication History period. Refer again to the visualization above for a visual aid.

Conduct the Negative Medication History Test

• If the fill date is within the eligible Intake Period (IPSD), and
• If there are no other fills within 105 days before the fill date, then
• The test confirms the Negative Medication History (TRUE), if not
• The test denies the Negative Medication History (FALSE)

Be patient. This part of the program takes about 20 minutes to run. If you think about what it is doing, it is comparing every row (147,000 rows) against the IPSD range, and for all the positive returns comparing each one to each of the the original 147,000 rows for the NMH test. Think about trying to guess the first 2 numbers on the bike lock. Anyway, it’s a lot. Go grab some coffee.

Code

Negative Medication History Test

Code

# Read the data from the IPSD test results
claims <- PBMClaims_MDDxTest

# Step 1: Select the desired columns and add ProjectId column
selected_columns <- claims %>%
  select(clientID, 
         PrimaryId,
         MemberID,
         genericnme, 
         GpiNumber,
         ProjectId_IPSD,
         IPSDate, 
         IPSDTestResult,
         ProjectId_MDD,
         MDDxDate,
         PrimaryDiagnosis,
         days_between_MDDx,
         MDDxTestResult
         ) 

# Step 2: Create copies of the table
original <- selected_columns
test <- selected_columns

# Step 3: Join the two tables based on the given conditions
joined_table <- original %>%
  #mutate(IPSDTestResult = as.logical(IPSDTestResult)) %>%
  filter(MDDxTestResult == "Positive MDDx") %>%
  inner_join(test, by = c("clientID", "genericnme"))

# Select desired variables
joined_table <- joined_table |> 
    select(clientID, 
         PrimaryId.x,
         MemberID.x,
         genericnme, 
         GpiNumber.x,
         ProjectId_IPSD.x,
         IPSDate.x, 
         IPSDTestResult.x,
         ProjectId_MDD.x,
         MDDxDate.x,
         PrimaryDiagnosis.x,
         days_between_MDDx.x,
         MDDxTestResult.x,
         ProjectId_IPSD.y,
         IPSDate.y
         ) 

# Step 4: rename dtefilled.x and dtefilled.y
joined_table <- joined_table %>%
  rename(NMHTestDate = IPSDate.y)

# Step 4 Rogue: format as date
joined_table <- joined_table %>%
  mutate(IPSDate.x = as.Date(IPSDate.x),
         NMHTestDate = as.Date(NMHTestDate))

# Step 4 Rogue 1: Compare IPSD and NMH dates
joined_table <- joined_table %>%
  mutate(comparison_result_NMH = case_when(
           IPSDate.x < NMHTestDate ~ "IPSD is prior to NMHTest date",
           NMHTestDate < IPSDate.x ~ "IPSD is after test NMHTest date",
           TRUE ~ NA_character_
         ))

# Step 5: Determine the number of days between IPSDate and NMHTestDate as an absolute value
joined_table <- joined_table %>% 
  mutate(days_between_NMH = abs(as.numeric(IPSDate.x - NMHTestDate)))

# Step 6: Determine "New Start" or "Positive Med History" based on days_between
joined_table <- joined_table %>%
  mutate(NMHTestResult = ifelse(days_between_NMH > 105, "New Start", "Positive Med History"))

# Do not write this, it is very large 25 million
# write.csv(joined_table, "./data/output/PBMClaims_NMHTest.csv")

PBMClaims_NMHTest <- joined_table

In lay terms

The code performs various operations on a dataset called “claims” (assumed to be in CSV format) related to pharmacy claims data with specific columns. Here’s a layman’s summary of the code:

1. Select desired columns from the claims data and add a new column called “ProjectId” with randomly generated 10-digit numbers.

2. Create copies of the table named “original” and “test” based on the selected columns.

3. Join the “original” and “test” tables based on specific conditions related to client ID and generic name.

4. Compare the “IPSDate” (date from the original table) with the “NMHTestDate” (date from the test table). Identify whether the IPSDate is before or after the NMHTestDate.

5. Calculate the number of days between the IPSDate and NMHTestDate as an absolute value.

6. Determine the “NMHTestResult” based on the number of days between IPSDate and NMHTestDate. If the number of days is greater than 105, the result is “New Start”; otherwise, it is “Positive Med History”.

7. Save the resulting joined table as a CSV file called “PBMClaims_NMHTest.csv”.

The overall purpose of the code seems to be to process and analyze pharmacy claims data, compare dates, and classify the results based on the number of days between certain dates (ChatGPT, personal communication, July, 2023).

Woo hoo! It worked!

We have now determined that there is:

1. A valid Index Prescription Start Date (IPSD)
2. A valid service for Major Depressive Disorder (MDDx)
3. A Negative Medication History (NMH)

Test for Continuous Enrollment

Description

Ok, three down, one to go. Awesome!

Now that we have confirmed IPSD, NMH, and MDDx, we will search for the member information in our enrollment records. We then determine if there was an eligible enrollment period associated with the prescription fill.

An prescription fill is eligible if the individual had continuous enrollment for -105 days before the IPSD, AND +231 days after the IPSD.

Procedure

Compare the Index Prescription Start Date against the member enrollment roster developed in the Alliance Progress Report ⁸. We use the full global members roster in this case because we are not filtering for active membership only. Individuals who are currently disenrolled are included in the dataset as well.

Code

Test for Continuous Enrollment

Code

# Import the original global members roster that is created in the Alliance Progress Report
GlobalMembers_orig <- read_xlsx("./data/data_original_glblmbrs_2023-05-01_globalMembersRoster.xlsx", sheet = "Sheet1")

# Create a working copy of the data and rename "x" to # "PrimaryId"
GlobalMembers <- GlobalMembers_orig |> 
  rename("PrimaryId.x" = x)

# Filter to rule out cases without a "Positive MDDx"

PBMClaims_CETest <- PBMClaims_NMHTest |> 
  filter(NMHTestResult == "New Start")

# Using the PBMClaims_MDDxTest table as a base, attach Global Members data by PrimaryId
PBMClaims_CETest1 <- merge(x = PBMClaims_CETest,
              y = GlobalMembers,
              by = "PrimaryId.x",
              all.x = TRUE) 


# Calculations for the prior enrollment period

# Convert the columns to Date format if they are not already in Date format
PBMClaims_CETest1$IPSDate.x <- as.Date(PBMClaims_CETest1$IPSDate.x)

PBMClaims_CETest1$bhhEffectiveDate <- as.Date(PBMClaims_CETest1$bhhEffectiveDate)

# Calculate the number of days between IPSDate and bhhEffectiveDate
PBMClaims_CETest1$days_between_CEPre <- as.numeric(PBMClaims_CETest1$IPSDate.x - PBMClaims_CETest1$bhhEffectiveDate)

# Add a column "CEPreTest" based on the condition
PBMClaims_CETest1 <- PBMClaims_CETest1 %>%
  mutate(CEPreTest = ifelse(days_between_CEPre >= 105, "Continuous Prior Enrollment", "No Continuous Prior Enrollment"))

# Calculations for the post prescription enrollment period

# Convert the columns to Date format if they are not already in Date format
PBMClaims_CETest1$IPSDate.x <- as.Date(PBMClaims_CETest1$IPSDate.x)

PBMClaims_CETest1$disenrollmentDate <- as.Date(PBMClaims_CETest1$disenrollmentDate)

# Calculate the number of days between IPSDate and bhhEffectiveDate
PBMClaims_CETest1$days_between_CEPost <- as.numeric(PBMClaims_CETest1$disenrollmentDate - PBMClaims_CETest1$IPSDate.x)

# Add a column "CEPostTest" based on the conditions
PBMClaims_CETest1 <- PBMClaims_CETest1 %>%
  mutate(CEPostTest = ifelse(days_between_CEPost >= 231 | is.na(disenrollmentDate), "Continuous Post Prescription Enrollment", "No Continuous Enrollment"))

# Add a finnal test column "CETestResult" based on the conditions
PBMClaims_CETest1 <- PBMClaims_CETest1 %>%
  mutate(CETestResult = ifelse(CEPreTest == "Continuous Prior Enrollment" & CEPostTest == "Continuous Post Prescription Enrollment",
                               "Continuous Enrollment", "No Continuous Enrollment"))

# Do not write, very large 7.9 million
# write.csv(PBMClaims_CETest1, "./data/output/PBMClaims_CETest.csv")

In lay terms

The R code performs the following operations:

1. Imports the original global members roster data from an Excel file into a variable called GlobalMembers_orig.
2. Creates a working copy of the GlobalMembers_orig data and renames the column “x” to “PrimaryId”.
3. Combines the PBMClaims_MDDxTest table with the GlobalMembers data based on the “PrimaryId” column, storing the result in PBMClaims_CETest1.
4. Filters out cases in PBMClaims_CETest1 that do not have a “Positive MDDx” value in the “MDDxTestResult” column.
5. Converts the “IPSDate” and “bhhEffectiveDate” columns in PBMClaims_CETest1 to the Date format if they are not already.
6. Calculates the number of days between the “IPSDate” and “bhhEffectiveDate” columns and stores the result in the “days_between_CEPre” column.
7. Adds a new column called “CEPreTest” to PBMClaims_CETest1 based on the condition that if the “days_between_CEPre” is greater than or equal to 105, the value is “Continuous Prior Enrollment,” otherwise it is “No Continuous Prior Enrollment.”
8. Converts the “IPSDate” and “disenrollmentDate” columns in PBMClaims_CETest1 to the Date format if they are not already.
9. Calculates the number of days between the “IPSDate” and “disenrollmentDate” columns and stores the result in the “days_between_CEPost” column.
10. Adds a new column called “CEPostTest” to PBMClaims_CETest1 based on the conditions that if the “days_between_CEPost” is greater than or equal to 231 or if the “disenrollmentDate” is NA (missing), the value is “Continuous Post Prescription Enrollment,” otherwise it is “No Continuous Enrollment.”
11. Adds a final column called “CETestResult” to PBMClaims_CETest1 based on the conditions that if both “CEPreTest” is “Continuous Prior Enrollment” and “CEPostTest” is “Continuous Post Prescription Enrollment,” the value is “Continuous Enrollment,” otherwise it is “No Continuous Enrollment.”

Overall, the code identifies cases with continuous enrollment, performs calculations related to enrollment periods, and determines the final enrollment status based on the results of the prior and post-prescription enrollment tests. (ChatGPT, personal communication, July, 2023).

Eligibility Model for Antidepressant Medication Management, Check!

This is the Real Data

Description

We have now determined that there is:

1. A valid Index Prescription Start Date,
2. A Negative Medication History,
3. A valid service for Major Depressive Disorder, and
4. A valid period of enrollment.

The resulting dataset presents a list of members with indicators for each of the respective tests. A person with a prescription fill that met each of the conditions is included, and will be compared with the VBP Quality Roster Data.

Code

Code

# Rule out cases without continuous enrollment
PBMClaims_AMMEligible <- PBMClaims_CETest1 |> 
  filter(CETestResult == "Continuous Enrollment")

# Select final variables
PBMClaims_AMMEligible <- PBMClaims_AMMEligible |> 
  select("PrimaryId.x",
         "clientID",
         "MemberID.x",
         "ahcccsId", 
         "ProjectId_IPSD.x", 
         "IPSDate.x", 
         "genericnme", 
         "ProjectId_MDD.x",
         "MDDxDate.x",
         "PrimaryDiagnosis.x",
         "days_between_MDDx.x",
         "MDDxTestResult.x",
         "ProjectId_IPSD.y",
         "NMHTestDate",
         "days_between_NMH",
         "NMHTestResult",
         "bhhEffectiveDate",
         "disenrollmentDate",
         "days_between_CEPre",
         "CEPreTest",
         "days_between_CEPost",
         "CEPostTest", 
         "CETestResult",
         "bhhShortname",
         "sex",
         "dob",
         "zipCode",
         "resCountyName")|> 
  filter(bhhShortname %in% c("CPIH", "TGC", "SBH", "CBI", "WYGC", "SHG", "MMHC", "LCBHC"))

# do not write, too big 3.9 million
# write.csv(PBMClaims_AMMEligible, "./data/output/PBMClaims_AMMEligible.csv")

In lay terms

The R code performs the following operations:

1. Filters the PBMClaims_CETest1 data to keep only cases with “Continuous Enrollment” based on the “CETestResult” column.
2. Selects a subset of variables from the filtered data, including columns like “PrimaryId,” “MemberID,” “IPSDate,” “NMHTestDate,” and others.
3. Filters the data further to include only cases where the “bhhShortname” column matches specific values, such as “CPIH,” “TGC,” “SBH,” and others.

Overall, the code selects cases that meet the criteria of continuous enrollment and specific “bhhShortname” values and prepares a subset of variables for further analysis or export (ChatGPT, personal communication, July, 2023).

Value-based Purchasing Quality Reports (VBP QR)

Now that we have a reliable dataset for the Real Data (adjudicated claims), we will introduce the Report Data (VBP Quality Roster). We will import and aggregate the Report Data, clean it up, and then compare it with Real Data.

Description

The VBP Roster comes to us from Health Choice, but the underlying data is analyzed and produced by a third party vendor, Cotivity.

The Alliance receives a separate report for each of the Alliance Providers. This procedure outlines how the VBP QR is input from the original report and transformed to actionable data.

The resulting table is a complete, cleaned roster of all of the individual member events, for all of the VBP Measures, for all of the Alliance Providers. It is then filtered to assess each measure individually.

This table is used to construct the VBP Quality Report Dashboard as well as the Alliance Progress Report

Procedure

The most recent Value-based Purchasing (VBP) Quality Roster (04-27-2023) for each Alliance Provider (AP) was aggregated. The Roster page from the excel data model was extracted from each of the individual reports and compiled into one aggregate data frame that contains the results of all Alliance Providers.

The VBP Report data is cumulative over the VBP measurement year 01-01-2023 to 12-31-2023 and has a 60 day claims lag, such that the 04-27-2023 VBP QR contains claims adjudicated through 02-28-2023.

The SubMeasureID variable was filtered for this study to only include the AMM2 measure, and a list of the unduplicated Member IDs.

Code

Import VBPQR Data

Code

# Import the unaltered VBP report, "Detail" sheet, as received from HCA
# 5/1/23 sheet = "Detail" was change by HCA to sheet = "Roster"
vbp_cbi   <-  read_xlsx("./data/VBPReports/Quality/vbpbhh_report_2023-05-31_94-2880847_Community_Bridges_HCA_BHH_VBP_Quality_Roster.xlsx", sheet = "Roster")
##  vbp_cbi <- vbp_cbi [,-1] 
##  colnames(vbp_cbi) <- c("BCBSAZ Health Choice" ,"...2", "...3", "...4", "...5", "...6", "...7", "...8", "...9")
vbp_cpih  <-  read_xlsx("./data/VBPReports/Quality/vbpbhh_report_2023-05-31_86-0215065_Change_Point_Integrated_Health_HCA_BHH_VBP_Quality_Roster.xlsx", sheet = "Roster")
vbp_lcbhc <-  read_xlsx("./data/VBPReports/Quality/vbpbhh_report_2023-05-31_86-0250938_Little_Colorado_Behavioral_Health_HCA_BHH_VBP_Quality_Roster.xlsx", sheet = "Roster")
vbp_mmhc  <-  read_xlsx("./data/VBPReports/Quality/vbpbhh_report_2023-05-31_86-0214457_Mohave_Mental_Health_HCA_BHH_VBP_Quality_Roster.xlsx", sheet = "Roster")
vbp_ph    <-  read_xlsx("./data/VBPReports/Quality/vbpbhh_report_2023-05-31_86-0206928_Polara_Health_HCA_BHH_VBP_Quality_Roster.xlsx", sheet = "Roster")
vbp_sbhs  <-  read_xlsx("./data/VBPReports/Quality/vbpbhh_report_2023-05-31_86-0290033_Southwest_Behavioral_Health_HCA_BHH_VBP_Quality_Roster.xlsx", sheet = "Roster")
vbp_shg   <-  read_xlsx("./data/VBPReports/Quality/vbpbhh_report_2023-05-31_86-0207499_Spectrum_Health_Group_HCA_BHH_VBP_Quality_Roster.xlsx", sheet = "Roster")
vbp_tgc   <-  read_xlsx("./data/VBPReports/Quality/vbpbhh_report_2023-05-31_86-0223720_The_Guidance_Center_HCA_BHH_VBP_Quality_Roster.xlsx", sheet = "Roster")

# Pro Tip
# if any of the tables pick up rogue columns...
# vbp_cbi <- vbp_cbi [,-1] 
# colnames(vbp_cbi) <- c("BCBSAZ Health Choice" ,"...2", "...3", "...4", "...5", "...6", "...7", "...8", "...9")

# Bind the Details sheet from all providers into one table
DataRaw_VBPQR_AllAPsCombined <- rbind(
  vbp_cbi,
  vbp_cpih,
  vbp_lcbhc,
  vbp_mmhc,
  vbp_ph,
  vbp_sbhs,
  vbp_shg,
  vbp_tgc
)

# write to csv
# date of file = date of VBP QR report
# write.csv(DataRaw_VBPQR_AllAPsCombined, "./data/output/2023-05-31_DataRaw_VBPQR_AllAPsCombined.csv")

Wrangle Said Data

Code

# create a safe copy of the original data
VBPQR_AllAPsCombined_Cleaned <- DataRaw_VBPQR_AllAPsCombined

# Filter out superfluous rows of nonsense data
VBPQR_AllAPsCombined_Cleaned <- VBPQR_AllAPsCombined_Cleaned |>  
  filter(`...2` != "NA")

# Set column names to headers, which get imported on row 1 #5/2/23 - updated from "6" 
colnames(VBPQR_AllAPsCombined_Cleaned) <- VBPQR_AllAPsCombined_Cleaned [1,] 

# Remove the first row of data that headers
VBPQR_AllAPsCombined_Cleaned <- VBPQR_AllAPsCombined_Cleaned[-1,]

# 5/1/23 - Create SubMeasureID
VBPQR_AllAPsCombined_Cleaned$`SubMeasure ID` <- substr(VBPQR_AllAPsCombined_Cleaned$Measure, 1, 3)

# write.csv(VBPQR_AllAPsCombined_Cleaned, "./data/output/VBPQualityRoster.csv")

Code

# create a duplicate at this phase to be used in later evaluation
VBPQR_AllAPsCombined_Cleaned2 <- VBPQR_AllAPsCombined_Cleaned |> 
  filter(`SubMeasure ID` == "AMM")

write.csv(VBPQR_AllAPsCombined_Cleaned2, "./data/output/VBPQualityRoster_AMM.csv")

# Isolate member ID for the validation
VBPQR_AllAPsCombined_Cleaned <- VBPQR_AllAPsCombined_Cleaned |> 
  filter(`SubMeasure ID` == "AMM") |> 
  select(`Member ID`)

# write.csv(VBPQR_AllAPsCombined_Cleaned, "./data/output/VBP_Validation.csv")

In lay terms

Import VBPQR data:

1. The code imports several VBP (Value-Based Purchasing) reports from different providers. These reports contain information about the quality of services related to behavioral health.
2. Each provider’s report is stored in a separate variable, such as vbp_cbi, vbp_cpih, vbp_lcbhc, etc. These variables represent the data tables within the reports.
3. The code reads the “Roster” sheet from each provider’s report file and stores the data in the corresponding variable.
4. All the provider data tables are then combined into one table called DataRaw_VBPQR_AllAPsCombined. This table contains the combined data from all providers.

Wrangle Said Data:

1. A copy of the combined data table, DataRaw_VBPQR_AllAPsCombined, is created for cleaning and manipulation.
2. Superfluous rows of irrelevant data are filtered out from the table.
3. The first row of the table, which contains headers, is used to set the column names for the table.
4. The first row of data (headers) is removed from the table.
5. A new column called “SubMeasure ID” is created by extracting the first three characters from the “Measure” column.
6. The cleaned and manipulated data table is saved as VBPQR_AllAPsCombined_Cleaned.

Wrangle VBPQR data 2:

1. Another copy of the cleaned data table, VBPQR_AllAPsCombined_Cleaned, is created for further evaluation.
2. This copy is filtered to include only rows where the “SubMeasure ID” is “AMM”.
3. The filtered data table is saved as VBPQR_AllAPsCombined_Cleaned2.
4. The original cleaned data table is further filtered to isolate the “Member ID” column for validation.
5. The filtered data table is saved as VBPQR_AllAPsCombined_Cleaned for validation purposes.

In summary, the code imports VBPQR reports from different providers, combines them into a single table, and performs data cleaning and manipulation to prepare the data for analysis and validation. The resulting data tables are saved for further use (ChatGPT, personal communication, July, 2023).

AMM Validation Data Modeling

The Validation Model for AMM is Conceptual and Logical. This model will help to define the systems surrounding measurement of AMM. This model also outlines a set of logical rules and structures of data to be used across agencies and measures.

NCQA HEDIS My2023 Validation Model for AMM

Description of Test Design

This model extracts data from multiple VBP Quality Reports and aggregates it them into a single dataset. It also collects adjudicated claims data from the Health Choice data warehouse.

The 2 datasets are then joined to create the Validation Matrix, comparing VBP cases against adjudicated claims.

The Validation Matrix is the table that will be used for evaluation! Phew! :)

Procedure

Report Data: VBP Quality Roster Reports

The VBPQR_AllAPsCombined_Cleaned data frame was loaded to the test model. The data was summarized, per member, by counting the instances of eligibility per member. This ultimately creates a vector of unduplicated Member IDs called VBP_Unduplicated.

Real Data: Adjudicated Claims

The AMMEligibilityTest claims data frame was loaded to the test model. It was then summarized by counting the instances of HEDIS My2022 eligible claims per member. This creates a vector of unduplicated Member IDs called AMMClaims_Unduplicated.

The Validation Matrix

The VBP_Unduplicated and the AMMClaims_Unduplicated are joined, and all rows of data from both variables are included, regardless of match. The resulting data frame is called Validation_Matrix.

• This table contains all of the unduplicated VBPQR MemberIDs, and all of the unduplicated Claims MemberIDs.

The data is assessed for cases where a VBP MemberID is validated against a Claims MemberID. - A positive result is called “Match”, and a negative result is called “NoMatch”

The Validation_Matrix is the table that will be used for evaluation! Phew! :)

Code

Code

# Extract just the member ID from the VBPQR AllAps, and than remove any duplicates
VBP_Unduplicated <- VBPQR_AllAPsCombined_Cleaned |> 
  group_by(`Member ID`) |> 
  rename("MemberID" = `Member ID`) |> 
  count()

# write.csv(VBP_Unduplicated, "./data/output/VBP_Unduplicated.csv")

Code

# Extract just the member ID from the PBMClaims_AMMEligible, and then remove any duplicates
AMMClaims_Unduplicated <- PBMClaims_AMMEligible |> 
  rename(MemberID = MemberID.x) |> 
  filter(MemberID != "NULL") |> 
  group_by(MemberID) |> 
  count()

# write.csv(AMMClaims_Unduplicated, "./data/output/AMMClaims_Unduplicated.csv")

Code

# Combine the VBP unduplicated members with the AMMClaims unduplicated, keeping all values, and matching them on MemberID whenever possible.
Validation_Matrix <- 
  merge(x = VBP_Unduplicated,
        y = AMMClaims_Unduplicated,
        by = "MemberID",
        all = TRUE) |> 
  rename("VBP" = n.x,
         "claims" = n.y) |> 
  mutate(Match = if_else((is.na(VBP) | is.na(claims)), "NoMatch", "Match"))

# write.csv(Validation_Matrix, "./data/output/ValidationMatrix.csv")

Summary

VBP Reports

• 8 VBP report details sheets were imported, merged and cleaned, creating DataRaw_VBPQR_AllAPsCombined with 4,424 observations of 9 variables.

• VBPQR_AllAPsCombined_Cleaned was created from the master to isolate instances of FUH7, and then select for Member ID, ultimately yielding 1,691 observations of MemberIDs.

Claims

• Pharmacy Records from HCA Pharmacy Benefit Manager were queried, including all claims for eligible AMM medications, for the date range including the IPSD and the NMH Tests. There were 196,520 records.
• Behavioral Health records were querried from the HCA behavioral health claims system for the date range including all eligible dates for a Major Depressive Diagnosis. There were 265,060 records.

Enrollment

• Enrollment data was queried from the HCA Global Members database for the entire range of possible enrollment and disenrollment dates. There were 367,940 members identified.

In lay terms

This R code is performing some data manipulation and analysis tasks. Let me break it down for you:

The first block of code, labeled “Modeling VBP Unduplicated,” is doing the following:

1. The dataset VBPQR_AllAPsCombined_Cleaned is being grouped by the column “Member ID.”
2. The column “Member ID” is then renamed to “MemberID.”
3. The number of occurrences of each unique “MemberID” is counted and stored in a new dataset called VBP_Unduplicated.
4. There is a commented out line (write.csv(…)) that suggests the intention to write the VBP_Unduplicated dataset to a CSV file.

The second block of code, labeled “Modeling Claims Unduplicated,” performs the following actions:

1. The dataset AMMEligibilityTest_AllEligible is filtered to exclude any rows where the “MemberID” is equal to “NULL.”
2. The dataset is then grouped by the “MemberID” column.
3. The number of occurrences of each unique “MemberID” is counted and stored in a new dataset called AMMClaims_Unduplicated.
4. There is a commented out line (write.csv(…)) that suggests the intention to write the AMMClaims_Unduplicated dataset to a CSV file.

The third block of code, labeled “Modeling Validation Matrix,” combines the previous two datasets (VBP_Unduplicated and AMMClaims_Unduplicated) to create a validation matrix:

1. The datasets VBP_Unduplicated and AMMClaims_Unduplicated are merged based on the common column “MemberID.” The resulting merged dataset is stored in Validation_Matrix.
2. The columns n.x and n.y (which represent the counts from the previous datasets) are renamed to “VBP” and “claims,” respectively.
3. A new column called “Match” is added to the Validation_Matrix dataset. The “Match” column is populated with the value “NoMatch” if either the “VBP” or “claims” values are missing (NA), and “Match” otherwise.
4. There is a commented out line (write.csv(…)) that suggests the intention to write the Validation_Matrix dataset to a CSV file.

In summary, this code takes two datasets (VBPQR_AllAPsCombined_Cleaned and AMMEligibilityTest_AllEligible), performs data manipulation and aggregation operations on them, and creates a validation matrix (Validation_Matrix) to determine matches and mismatches based on the “MemberID” column. The intention is to potentially write the resulting datasets to CSV files for further analysis or storage.

Results

Validating Report Data using Real Data

In order to conduct the validation test, all records from VBP_Unduplicated are included, while Claims_Unduplicated records are only included if they have a positive match with VBP_Unduplicated on the variable Member ID.

Validation Matrix Results

The Validation Matrix dataset is a list of all distinct member IDs in the context of claims and VBP reports.

Validation

The cases when a member identified on the Health Choice VBP Quality Roster is also identified in adjudicated claims were as follows:

• Valid (“Match”): 266 of the 1,691 members on the VBP Quality Roster were also found among eligible Health Choice pharmacy claims.
• Not-Valid (“No Match”): 1,425 of the 1,691 members on the VBP Quality Roster were not found among eligibile Health Choice pharmacy claims.

Compliance

The compliance status of cases on the Health Choice VBP Report were as follows.

• Non-Compliant: 703 cases were not found to have the expected duration of antidepressant medication treatment.

• Compliant: 988 cases were found to have the expected duration of antidepressant medication treatment.

Validation by Compliance:

The matrix that results from combining the validation status and the compliance status of a given member provides the following results. Compliance status is provided by the VBP Quality Roster. The Alliance has not created a model for the assessment of compliance, yet.

• Matched, NonCompliant (MNC): 36
  • A member that is found in both the VBP Quality Roster AND found in Health Choice claims, who DID NOT receive the expected duration of antidepressant medication treatment
• Matched, Compliant (MC): 230
  • A member that is found in both the VBP Quality Roster AND found in Health Choice claims, who DID indeed receive the expected duration of antidepressant medication treatment
• NonMatched, NonCompliant (NMNC): 667
  • A member that is found in the VBP Quality Roster, BUT NOT found in Health Choice claims, who DID NOT receive the expected duration of antidepressant medication treatment
• NonMatched, Compliant (NMC): 758
  • A member that is found in the VBP Quality Roster, BUT NOT found in Health Choice claims, who DID indeed receive the expected duration of antidepressant medication treatment

Analysis

The Pearson’s Chi-Square test can be used to confirm if there is a significant difference between the expected result and the observed result in this comparison.

Chi Square test was run at alpha = .05. There was a statistically significance between the groups (X^2 (1, N=1691) = 90.633, p<.001), indicating that the groups in the Validation Matrix are disproportionately impacted.

We also found that only about 16% of the cases that were identified by the VBP Quality roster were also found in adjudicated claims. In other words, only 16% of the VBP Quality Roster cases were valid.

Code

# Create a 2 x 2 matrix for the chi square test

# transform for Chi Square test
Comp_ChiSq <- Compliance |> 
    select(Match, Gap_Status) |> 
  group_by(Match, Gap_Status) |>
  #mutate(Numerator = as.character(Numerator)) |> 
  count() |> 
  pivot_wider(names_from = "Gap_Status",
              values_from = "n")
  # mutate(`0` = as.double(`0`),
  #        `1` = as.double(`1`))

# write.csv(Comp_ChiSq, "./data/output/Comp_ChiSq.csv")

chisq.test(Comp_ChiSq[-1], correct = FALSE)

    Pearson's Chi-squared test

data:  Comp_ChiSq[-1]
X-squared = 102.17, df = 1, p-value < 2.2e-16

Code

# Create chi square column chart

ChiSquare_Barchart <- Compliance |> 
  select(Match, Gap_Status) |> 
  group_by(Match, Gap_Status) |>
  mutate(Numerator = if_else(Gap_Status == "OPEN", "NonCompliant", "Compliant")) |> 
  count() |> 
  ggplot(aes(fill=Match, x=Gap_Status, y=n)) +
  geom_col() +
  scale_fill_manual(values = c(Match = "#b60ef9", NoMatch = "#520096"),
                    name = "Validation",
                    labels = c("VBP and Claims", "VBP and No Claims")) +
  labs(title = "VBP Report Member Event Validation",
       subtitle = "VBP Quality Roster validation against adjudicated claims",
       caption = "*From claims adjudicated through April 27, 2023") +
  ylab("Number of Distinct Members") +
  xlab("Compliance to expected AMM treatment duration (115 days)") +
  scale_x_discrete(labels = c("115 days Tx (Compliant)", " Less than 115 days (Non-Compliant)")) +
  theme(
    axis.title.y = element_text(vjust = 2),
    plot.subtitle = element_text(face = "italic"),
    plot.caption = element_text(face = "bold.italic", hjust = 0, vjust = -1)
  ) +
  annotate("text", x = 1, y = 800, label = "p < .001", color = "white", fontface = "bold")

# Save the plot as a PNG file with a transparent background
ggsave("./data/output/chisquared_transparentbackground.png", width = 7, height = 5, dpi = 300, bg = "transparent")

# ggsave("./data/output/AMMchisquare_barchart.png", width = 7, height = 5, dpi = 300)

# ggsave("C:/Users/RyanK/OneDrive - The NARBHA Institute/R_Studio/progressReport/images/20230630_AMMchisquare_barchart.png", width = 7, height = 5, dpi = 300)

ChiSquare_Barchart

Discussion

Eligibility Model

The process of determining eligible cases for the AMM measurement is very complex. We hope that we were able to adequately describe the process and the nature of the resulting dataset.

After determining the eligible cases, the process of validating that list of members against the VBP Quality Roster is fairly straight forward, and followed an almost identical process to the FUH7 Validation.

The evaluation of the aggregated Alliance Provider VBP Quality Rosters identified 1,691 eligible members. However, only 16% of those (266 members) could be validated by matching the VBP QR record with an eligible claim.

Conclusions

Our goal is accuracy and transparency. With this in mind, The Alliance should keep a close eye on this measure. With the 1st quarter of data for the 2023 measurement year in the books, it is a concern that there are so few valid cases in the VBP Quality Report. This data will be reported to Health Choice leadership at our quarterly joint operating meeting.

The difference between the validation groups was statistically significant. It is also easily visualized. The implications of this data are considerable. The current AMM performance score for the Alliance is 58.43%, or +1.14 percentage points above the mean. When that score is adjusted to only include cases that can be validated by matching with an eligible claim (the light purple), that performance scores rises to 86.5%, or +30.93 percentage points.

This is significant impact on the performance measurement of Alliance Providers. We recommend further investigation into the data and processes used by Health Choice. In the meantime, it is also recommended to research provider tools and innovations to improve the overall performance of the network while we continue to assess data validation.

Quality Reviews

1. Internal Alliance Leadership: 06-29-2023
2. The Narbha Institute Leadership: 07-03-2023
3. Internal Alliance Leadership: 07-06-2023
4. Alliance-Health Choice Joint Operating Committee:

Appendix

All Alliance Performance Measures

⁹

Create a GPI to NDC crosswalk

Because HCA categorizes their PBM claims in terms of Generic Product Identifier (GPI), but NCQA does theirs in terms of National Drug Code (NDC), we have to crosswalk the eligible AMM medications from NDC to their corresponding GPI codes. Fortunately, it’s totally not a pain in the neck to backwards engineer this at all, lol.

Description

The HEDIS My2023 Medication List Names identifies the eligible medications by providing the respective NDC numbers¹⁰. The AHCCCS Preferred Drug List is used to match NDC numbers to GPI numbers based on medication name¹¹.

Procedure

In order to create a cross walk that ties the NDC codes from NCQA to the GPI codes from Health Choice, we took the following steps:

• Import the My2023 Medication to Code data from the My2023 Medication List Directory¹²
• Filter the data to only include Antidepressant Medications
• Rename some columns to provide consistency across tables
• Import the AHCCCS Preferred Drug List ¹³. A side by side list of NDC and GPI codes is incredibly difficult to come by, and so we are using this list from 2019. Fortunately NDC and GPI are very stable.
• Rename some columns to provide consistency across tables.
• Filter Therapeutic Class for Antidepressant Other, Antidepressant SSRI
• Merge the NCQA dataset with the AHCCCS dataset by matching on the the NDC variable.

Code

Code

# Covert NDC to GPI

#import the My2023 Medications to Code dataset
Med_To_NDC <- read.csv("./data/My2023MedicationToCode.csv")

Med_To_NDC_AMM <- Med_To_NDC |> 
  filter(Medication.List.Name == "Antidepressant Medications") |> 
  select(
    Medication.List.Name,
    Code,
    Generic.Product.Name
  )

colnames(Med_To_NDC_AMM)[colnames(Med_To_NDC_AMM) == "Code"] <- "NDC_NationalDrugCode"

colnames(Med_To_NDC_AMM)[colnames(Med_To_NDC_AMM) == "Medication.List.Name"] <- "MedicationList"

colnames(Med_To_NDC_AMM)[colnames(Med_To_NDC_AMM) == "Generic.Product.Name"] <- "GenericProductName"

# Import AHCCCS Preferred drug list
NDC_To_GPI <- read.csv("./data/AHCCCS_PreferredDrugListChangesFor_08012019.csv")

colnames(NDC_To_GPI)[colnames(NDC_To_GPI) == "Therapeutic.Class...Market.Basket"] <- "TherepeuticClass"

colnames(NDC_To_GPI)[colnames(NDC_To_GPI) == "National.Drug.Code..NDC..MediSpan"] <- "NDC_NationalDrugCode"

colnames(NDC_To_GPI)[colnames(NDC_To_GPI) == "MediSpan.Generic.Product.Indicator..GPI."] <- "GPI_GenericProductIdentifier"


NDC_To_GPI_AMM <- NDC_To_GPI |> 
  filter(c(TherepeuticClass == "ANTIDEPRESSANTS, OTHER" | TherepeuticClass ==  "ANTIDEPRESSANTS, SSRIs")) |> 
  select(NDC_NationalDrugCode,
         GPI_GenericProductIdentifier,
         TherepeuticClass)

# merge then filter for only AHCCCS preferred medications

# So these are all of the ones from NCQA that AHCCCS has on thier list. And now we have our crosswalk to address claims. 

My2023NDCtoGPICrosswalk <- merge(
  x = Med_To_NDC_AMM,
  y = NDC_To_GPI_AMM,
  by = "NDC_NationalDrugCode",
  all.y = TRUE
) |> 
  select(
    MedicationList,
    TherepeuticClass,
    NDC_NationalDrugCode,
    GPI_GenericProductIdentifier,
    GenericProductName
  ) |> 
  na.omit()

write.csv(My2023NDCtoGPICrosswalk, "./data/output/My2023NDCtoGPICrosswalk.csv")

concatenated_values_GPI <- paste0("'",    My2023NDCtoGPICrosswalk$GPI_GenericProductIdentifier, "'", collapse = ", ")

Create an AMM Member Follow Up List

Description

The Validation Matrix and the original dataset for PBM Claims are combined to form a member list with an indicator for VBP, claims, validation, and the associated PBM claims variables.

Procedure

The Validation_Matrix was transformed and rejoined with AMMEligibilityTest to create a table that will be used to generate Member Follow Up Lists. This new table is called MemberFollowUpList.

Code

Code

# Using the copy of VBPQR_AllAPsCombined_Cleaned we made above, rename MemberID to match

PBMClaims_AMMEligible <- PBMClaims_AMMEligible |> 
  rename(MemberID = MemberID.x)

# Rejoin the matched and non-matched MemberIDs to their claims data
MemberFollowUpList <- 
   merge(x = Validation_Matrix,
        y = PBMClaims_AMMEligible,
        by = "MemberID",
        all = TRUE) |>
  drop_na(Match)

# Do not write, too big, 3.9 million
# write.csv(MemberFollowUpList,"./data/output/AMM_MemberFollowUpList.csv")

Evaluate Validated Data

Description

Some exploratory analyses were conducted on the resulting data from this model. The results are included here for further reference, and we invite any innovations that spring from its review.

Top 10 Label Names

Code

# Summarize top ten label names

MemberFollowUpList |> 
  filter(!is.na(genericnme)) |>
  group_by(genericnme) |> 
  summarise(percent = n() / nrow(MemberFollowUpList) * 100) |>
  arrange(desc(percent)) |>
  head(10) |>
  kable()

genericnme	percent
SERTRALINE HCL TAB 100 MG	17.069079
VENLAFAXINE HCL CAP ER 24HR 75	10.968159
VENLAFAXINE HCL CAP ER 24HR 15	10.112130
FLUOXETINE HCL CAP 40 MG	10.091155
BUPROPION HCL TAB 100 MG	9.114629
TRAZODONE HCL TAB 50 MG	6.407451
TRAZODONE HCL TAB 100 MG	6.188917
FLUOXETINE HCL CAP 20 MG	4.887267
TRAZODONE HCL TAB 150 MG	4.854844
BUPROPION HCL TAB ER 24HR 300	4.430453

Code

# Visualize top 10 label names

MemberFollowUpList |>
  filter(!is.na(genericnme)) |>
  group_by(genericnme) |>
  summarise(percent = n() / nrow(MemberFollowUpList) * 100) |>
  filter(percent >= 1) |>
  arrange(genericnme) |>
  ggplot(aes(x = percent, y = reorder(genericnme, percent))) +
  geom_bar(stat = "identity", fill = "steelblue") +
  ylab(NULL) +
  xlab("Percentage of Total Prescription Fills") +
  ggtitle("Horizontal Bar Chart of Label Names with at least 1%") +
  theme_minimal() +
  theme(axis.text.y = element_text(angle = 0, hjust = 1, margin = margin(r = 5)),
        axis.title.y = element_blank(),
        plot.title = element_text(hjust = 1.0),
        axis.text.x = element_text(hjust = 0),
        axis.text.y.right = element_text(hjust = 0))

Prescription Fill Trend

Code

# Evaluate date of service

MemberFollowUpList |>
  mutate(IPSDate.x = as.Date(IPSDate.x)) |>
  mutate(month = floor_date(IPSDate.x, "month")) |>
  filter(month >= as.Date("2022-04-01") & month <= as.Date("2023-03-31")) |>
  count(month) |>
  ggplot(aes(x = month, y = n)) +
  geom_line() +
  geom_vline(xintercept = as.Date("2022-10-01"), linetype = "dashed", color = "#520096") +
  annotate("text", x = as.Date("2022-10-01"), y = max(MemberFollowUpList$n), label = "End RBHA Contract", 
           vjust = -10.0, hjust = -0.025, color = "#520096") +
  scale_x_date(date_labels = "%b %Y", date_breaks = "1 month") +
  scale_y_continuous(labels = comma) +
  ylab("Number of Fills") +
  xlab(NULL) +
  ggtitle("Frequency of Antidepressant Medication Fills over Time") +
  theme_minimal() +
  theme(axis.text.x = element_text(angle = 45, hjust = 1))

Record of the R Studio Environment, files and objects

Code

# Install 'knitr' package if not already installed
if (!require(knitr)) {
  install.packages("knitr")
}

Loading required package: knitr

Warning: package 'knitr' was built under R version 4.2.2

Code

library(knitr)

# Get a list of all objects in the environment
objects <- ls()

# Initialize an empty list to store object details
object_details <- list()

# Iterate over each object
for (object_name in objects) {
  # Get the object
  object <- get(object_name)
  
  # Get the number of observations and variables
  num_observations <- if (is.data.frame(object)) nrow(object) else NA
  num_variables <- if (is.data.frame(object)) ncol(object) else NA
  
  # Check if observations are NA
  if (!is.na(num_observations)) {
    # Create a data frame with object details
    object_info <- data.frame(Object = object_name,
                              Observations = num_observations,
                              Variables = num_variables)
    
    # Add the object details to the list
    object_details[[object_name]] <- object_info
  }
}

# Combine all object details into a single data frame
all_object_details <- do.call(rbind, object_details)

# Format the table using kable
formatted_table <- kable(all_object_details, format = "markdown", align = "c")

# Print the formatted table
print(formatted_table)

|                              |            Object             | Observations | Variables |
|:-----------------------------|:-----------------------------:|:------------:|:---------:|
|AMMClaims_Unduplicated        |    AMMClaims_Unduplicated     |     1280     |     2     |
|claims                        |            claims             |    492543    |    28     |
|Comp_ChiSq                    |          Comp_ChiSq           |      2       |     3     |
|Compliance                    |          Compliance           |     1691     |    13     |
|DataRaw_VBPQR_AllAPsCombined  | DataRaw_VBPQR_AllAPsCombined  |     4424     |     9     |
|filtered_table                |        filtered_table         |    147649    |    22     |
|GlobalMembers                 |         GlobalMembers         |    367940    |    13     |
|GlobalMembers_orig            |      GlobalMembers_orig       |    367940    |    13     |
|InLineCode                    |          InLineCode           |      1       |    10     |
|joined_table                  |         joined_table          |   25859394   |    18     |
|MDDxClaims                    |          MDDxClaims           |    265060    |    17     |
|MDDxClaimsTest                |        MDDxClaimsTest         |    103515    |     5     |
|MemberFollowUpList            |      MemberFollowUpList       |   3904657    |    31     |
|original                      |           original            |    492543    |    13     |
|PBMClaims                     |           PBMClaims           |    196520    |    21     |
|PBMClaims_AMMEligible         |     PBMClaims_AMMEligible     |   3903232    |    28     |
|PBMClaims_CETest              |       PBMClaims_CETest        |   7914160    |    18     |
|PBMClaims_CETest1             |       PBMClaims_CETest1       |   7914160    |    35     |
|PBMClaims_IPSDTest            |      PBMClaims_IPSDTest       |    196520    |    22     |
|PBMClaims_MDDxMerge           |      PBMClaims_MDDxMerge      |    593514    |    26     |
|PBMClaims_MDDxTest            |      PBMClaims_MDDxTest       |    492543    |    28     |
|PBMClaims_NMHTest             |       PBMClaims_NMHTest       |   25859394   |    18     |
|selected_columns              |       selected_columns        |    492543    |    13     |
|test                          |             test              |    492543    |    13     |
|Validation_Matrix             |       Validation_Matrix       |     2705     |     4     |
|vbp_cbi                       |            vbp_cbi            |     393      |     9     |
|vbp_cpih                      |           vbp_cpih            |     677      |     9     |
|vbp_lcbhc                     |           vbp_lcbhc           |     214      |     9     |
|vbp_mmhc                      |           vbp_mmhc            |     1113     |     9     |
|vbp_ph                        |            vbp_ph             |     385      |     9     |
|vbp_sbhs                      |           vbp_sbhs            |     1176     |     9     |
|vbp_shg                       |            vbp_shg            |     223      |     9     |
|vbp_tgc                       |            vbp_tgc            |     243      |     9     |
|VBP_Unduplicated              |       VBP_Unduplicated        |     1691     |     2     |
|VBPQR_AllAPsCombined_Cleaned  | VBPQR_AllAPsCombined_Cleaned  |     1691     |     1     |
|VBPQR_AllAPsCombined_Cleaned2 | VBPQR_AllAPsCombined_Cleaned2 |     1691     |    10     |

Footnotes

1. Health Choice Arizona. (2023). Value Based Purchasing Quality Roster

2. Kivela, J.R. (2023). Value Based Purchasing Data Validation Model. The Northern Arizona Regional Behavioral Health Alliance.

3. NCQA. (2023). HEDIS and Performance Measurement. https://www.ncqa.org/hedis/

4. NCQA. (2023). HEDIS Measures and Technical Resources. https://www.ncqa.org/hedis/measures/

5. Kivela, J.R. (2023). Value Based Purchasing Data Validation Model. The Northern Arizona Regional Behavioral Health Alliance.

6. CRISP-DM help overview. (August 17, 2021). https://www.ibm.com/docs/en/spss-modeler/saas?topic=dm-crisp-help-overview.

7. Kivela, J.R. (June, 2023). Value Based Purchasing Report. The Northern Arizona Regional Behavioral Health Alliance.

8. Kivela, J.R., McMillian, J., Tewa, V. (June 2023). The Alliance Progress Report. The Northern Arizona Regional Behavioral Health Alliance.

9. Kivela, J.R. (June, 2023). Value Based Purchasing Report. The Northern Arizona Regional Behavioral Health Alliance.

10. NCQA. (2023). HEDIS Measures and Technical Resources. https://www.ncqa.org/hedis/measures/

11. Arizona Health Care Cost Containment System.AHCCCS Drug List (Effective April 1, 2023). https://www.azahcccs.gov/Resources/Downloads/PharmacyUpdates/AHCCCS_DRUG_LIST_04012023.pdf

12. NCQA. (2023). HEDIS Measures and Technical Resources. https://www.ncqa.org/hedis/measures/

13. Arizona Health Care Cost Containment System. AHCCCS Drug List (Effective April 1, 2023). https://www.azahcccs.gov/Resources/Downloads/PharmacyUpdates/AHCCCS_DRUG_LIST_04012023.pdf